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Abstract 

Continued  advances  in  display  hardware,  computing  power,  net¬ 
working,  and  rendering  algorithms  have  all  converged  to  dramati¬ 
cally  improve  large  high-resolution  display  capabilities.  We  present 
a  survey  on  prior  research  with  large  high-resolution  displays.  In 
the  hardware  configurations  section  we  examine  systems  includ¬ 
ing  multi-monitor  workstations,  reconfigurable  projector  arrays,  and 
others.  Rendering  and  the  data  pipeline  are  addressed  with  an 
overview  of  current  technologies.  We  discuss  many  applications  for 
large  high-resolution  displays  such  as  automotive  design,  scientific 
visualization,  control  centers,  and  others.  Quantifying  the  effects 
of  large  high-resolution  displays  on  human  performance  and  other 
aspects  is  important  as  we  look  toward  future  advances  in  display 
technology  and  how  it  is  applied  in  different  situations.  Interacting 
with  these  displays  brings  a  different  set  of  challenges  for  HCI  pro¬ 
fessionals,  so  an  overview  of  some  of  this  work  is  provided.  Finally, 
we  present  our  view  of  the  top  ten  greatest  challenges  in  large  high- 
resolution  displays. 
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1  Introduction 

With  technological  advances,  large  high-resolution*  displays  are  be¬ 
coming  prevalent  in  many  fields.  From  multi-monitor  configura¬ 
tion  to  tiled  LCD  panels  to  projection-based  seamless  displays,  re¬ 
searchers  have  been  constructing  large  displays  with  various  hard¬ 
ware  configurations.  Two  common  features  of  such  displays  are  in¬ 
creased  physical  size  and  higher  resolution.  Researchers,  however, 
have  encountered  numerous  technological  difficulties  with  building 
large-format  high-resolution  displays,  especially  with  tiling  com¬ 
mercially  available  projectors  into  a  seamless  display  landscape.  Ac¬ 
cordingly,  they  have  proposed  a  number  of  techniques  to  address 
those  fundamental  problems  and  matured  the  construction  of  large 
high-resolution  displays. 

Meanwhile,  researchers  have  been  designing  and  implementing 
software  toolkits  that  support  large  high-resolution  displays.  A  sin¬ 
gle  PC  workstation  is  far  from  sufficient  to  drive  a  display  made  up 
of  more  than  ten  monitors  or  projectors.  It  is  now  common  to  use  a 
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PC  cluster,  a  set  of  computers  connected  via  a  high-speed  network, 
for  rendering  on  large  high-resolution  displays.  Cluster  rendering 
algorithms  and  systems  have  been  interesting  to  computer  scientists, 
and  distributed  rendering  software  as  well  as  data  streaming  archi¬ 
tectures  have  been  made  available  to  the  public. 

As  an  emerging  technology,  large  high-resolution  displays  have 
been  widely  applied  in  various  domains.  The  increasing  popularity 
of  large  high-resolution  displays  [37]  is  paralleled  by  booming  re¬ 
search  efforts  in  addressing  a  fundamental  question:  how  do  users 
benefit  from  increased  size  and  resolution?  Many  intuitively  believe 
that  large  displays  automatically  outperform  small  ones.  It  is  desir¬ 
able,  however,  to  understand  why  increased  size  and  resolution  are 
advantageous,  and  how  we  benefit  from  large  high-resolution  dis¬ 
plays  in  accomplishing  general  or  domain- specific  tasks.  Quantita¬ 
tive  and  qualitative  experiments  have  been  conducted,  gathering  em¬ 
pirical  evidence  to  demonstrate  the  relationship  between  the  chang¬ 
ing  visual  effects  afforded  by  emerging  technologies  and  users’  pro¬ 
ductivity  and  performance  in  collaborative  and  individual  work. 

As  integral  components  in  a  computing  environment,  user  inter¬ 
faces  and  interaction  techniques  are  constantly  drawing  attention  of 
researchers.  Many  traditional  user  interfaces  and  interaction  tech¬ 
niques  become  awkward  or  next  to  impossible  to  operate  on  large 
high-resolution  displays  [63].  Researchers  have  been  attempting  to 
modify  or  extend  existing  interface  metaphors  for  large-format  dis¬ 
plays.  They  have  also  been  creating  novel  interface  techniques  that 
scale  well  on  large  displays.  In  addition,  solutions  for  input  tech¬ 
nologies  other  than  mouse  and  keyboard  have  been  proposed  to  in¬ 
teract  with  large-format  displays.  Pen-based  techniques,  laser  point¬ 
ers,  and  gestures  are  just  a  few  examples. 

Many  challenges  remain  for  large  high-resolution  displays  in 
various  aspects,  and  research  on  large  high-resolution  displays  is 
very  active  and  progressing  rapidly,  partially  pushed  by  industry 
(e.g.  high-definition  TV  systems)  [63].  We  survey  the  literature 
on  large  high-resolution  displays,  covering  research  aspects  of  hard¬ 
ware  configurations  (Section  2),  rendering  and  streaming  software 
(Section  3),  applications  of  large  displays  (Section  4),  visual  effects 
and  human  performance  (Section  5),  and  user  interfaces  and  interac¬ 
tion  techniques  (Section  6).  We  conclude  by  proposing  ten  research 
challenges  for  utilizing  and  interacting  with  large  high-resolution 
displays.  By  specifying  these  challenges,  we  hope  to  inspire  orig¬ 
inal  future  research. 

2  Hardware  Configurations 
2.1  CAVE  and  Derivatives 

Among  the  first  large-format  display  systems  to  receive  widespread 
use  was  the  CAVE™ (CAVE  Automatic  Virtual  Environment),  a 
projection-based  Virtual  Reality  (VR)  system  that  surrounds  viewers 
in  an  immersive  environment  with  four  or  more  large  display  walls. 

The  word  resolution  historically  means  the  density  of  pixels  on  the 
screen,  usually  in  terms  of  dots  per  inch  (DPI)  [13].  However,  it  is  becoming 
common  practice  to  refer  to  resolution  as  the  number  of  pixels  on  a  display, 
especially  when  people  use  the  term  “high-resolution  displays.” 
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A  CAVE  system  typically  arranges  four  10  ft  xlO  ft  screens  in  a 
cube  made  up  of  three  rear-projection  screens  for  walls  and  a  front- 
projection  screen  for  the  floor  [23].  Five- wall  [134]  and  six- wall 
configurations  [99]  exist  as  well,  but  they  require  special  screens  that 
can  support  the  weight  of  users  and/or  movable  screens  to  enable  en¬ 
try  into  the  facility.  One  concern  with  constructing  CAVE  systems  is 
the  space  for  the  optical  path  between  projectors  and  screens.  Most 
implementations  fold  each  optical  path  with  at  least  one  mirror  in 
order  to  shrink  the  footprint  of  the  total  system.  Typically,  all  users 
wear  stereo  shutter  glasses  and  one  user  wears  a  head  tracker  with  six 
degrees  of  freedom  (DOF).  This  enables  all  users  to  see  stereo  im¬ 
agery,  although  only  the  tracked  user  will  have  a  correct  perspective 
projection.  Tracked  3D  input  devices  afford  multiple  DOF  interac¬ 
tion;  data  gloves  and  joysticks  are  commonly  employed  in  immer¬ 
sive  virtual  environments.  Factors  including  an  increased  display 
area,  a  wider  field  of  view  and  field  of  regard,  multiple  DOF,  and 
stereopsis  combine  to  generate  an  increased  level  of  immersion  [89]. 

The  current  highest  resolution  visualization  room  in  the 
world  [34]  (Colorplate  Figure  Cl)  has  been  developed  by  Fakespace 
Systems  [33]  and  is  installed  at  the  Advanced  Simulation  and  Com¬ 
puting  (ASC)  Program’s  site  at  Los  Alamos  National  Laboratory. 
The  immersive  viewing  environment  provides  a  43  million  pixel  dis¬ 
play  across  five  rear-projected  screens  (three  walls,  floor,  and  ceil¬ 
ing).  Nine  tiles  compose  the  front  wall  and  six  tiles  for  each  re¬ 
maining  screen.  A  total  of  33  stereoscopic  digital  projectors  are 
seamlessly  tiled  to  produce  continuous  images  across  the  display 
environment  which  measures  15  ft  wide  by  10  ft  deep  by  12  ft  high. 

2.2  Multi-Monitor  Desktop 

Multi-monitor  (or  “multimon”  in  other  literature)  is  an  increasingly 
popular  configuration  in  businesses  and  homes  to  extend  a  standard 
desktop  PC  with  more  screen  real  estate.  Modem  operating  systems 
such  as  Windows,  Mac  OS  X  and  Linux  offer  plug-and-play  capabil¬ 
ity,  and  dual-head  video  output  is  common  on  even  a  modest  graph¬ 
ics  card.  With  PCI®  and  PCI  Express®  SLI™graphics  cards,  hav¬ 
ing  multiple  heads  is  also  possible.  Multi-monitor  configurations  are 
easy  to  configure,  without  demanding  expertise  in  computer  science 
or  related  subjects.  Figure  la  shows  an  example  of  a  multi-monitor 
setup  for  desktop  computing.  Figure  lb  and  Colorplate  Figure  C2 
show  a  tiled-LCD  multi-monitor  desktop  possibility. 


(b) 

Figure  1:  Desktop  configurations:  (a)  dual  monitor  and  (b)  tiled-LCD 
multi-monitor  desktop  display  (Images  courtesy  Ball  [4]). 

Multi-monitor  setups  are  receiving  more  attention  in  the  research 
community.  For  example,  bezel  issues  (i.e.,  edges  of  the  tiles  break¬ 
ing  up  the  continuity  of  the  large  display)  have  been  examined  in 
[70,  114].  Usability  issues  of  multi-monitor  setups  for  multi-tasking 
have  been  explored  in  [25, 4, 51].  Researchers  also  attempted  to  con¬ 
struct  projection-based  multi-monitor  displays.  Starkweather  et  al. 
created  a  curved  desktop  called  DSHARP  [109]  using  DLP  projec¬ 
tors  and  parabolic  mirrors.  Bishop  and  Welch  [12]  created  a  “desk¬ 
top”  environment  that  used  projections  on  the  wall  to  alleviate  bezel 
and  ergonomic  issues. 

2.3  Tiled  LCD  Panels 

Tiled  LCD  panels  are  sets  of  LCD  displays  arranged  in  a  2D  array. 
The  array  can  be  arranged  flat  like  a  wall  display  (Figure  2a  and 


Colorplate  Figure  C3),  flat  like  a  table  (Figure  2b  and  Colorplate 
Figure  C4),  curved,  or  in  other  configurations.  The  combined  pixel 
count  across  the  arrays  can  reach  into  the  100  million  pixel  range. 
For  example,  the  Electronic  Visualization  Laboratory,  University  of 
Illinois  (EVL-UIC)  has  developed  a  large  100  MPixel  display  called 
LambdaVision  (Figure  2a  and  Colorplate  Figure  C3)  [94,  95]  and 
NASA  has  developed  the  Hyperwall  which  contains  49  LCD  panels 
tiled  in  a  7x7  array  [103]. 

Some  advantages  for  tiled  LCD  panels  are:  (1)  they  are  easier  to 
align  and  color  correct  than  projectors;  (2)  they  are  less  expensive 
than  projectors,  which  use  expensive  bulbs  with  relatively  short  life¬ 
times;  and  (3)  they  take  less  space  (no  throw  distance  needed).  A 
disadvantage  is  the  array  has  borders  between  each  tile  (it  affects 
displaying  of  text,  but  is  not  so  bad  for  imagery). 

The  large  tiled  LCD  panel  displays  offer  a  variety  of  uses  due 
to  their  high  pixel  counts.  Ultra-high  resolution  imagery  such  as 
geospatial  data  can  be  shown  (and  interacted  with)  in  one  contiguous 
display.  Multiple  display  content  (e.g.,  graphics,  desktop  content, 
video,  and  imagery)  can  be  displayed  all  at  once  on  different  parts 
of  the  display. 

EVL-UIC  has  developed  a  table-top  tiled  LCD  display  called  the 
LambdaTable  [62,  93]  (Figure  2b  and  Colorplate  Figure  C4).  The 
advantage  of  the  table-top  format  is  that  it  is  familiar  to  many  users 
who  prefer  a  sandbox  as  a  metaphor:  every  user  can  see  the  physical 
shape  in  a  sandbox  and  can  move  to  a  location  to  modify  the  surface. 
The  tangible  nature  of  the  interaction  with  the  data  is  a  big  advan¬ 
tage.  The  table-top  nature  makes  the  system  practical  for  many  users 
to  collaborate  in  an  application. 


Figure  2:  Tiled  LCD  panels:  (a)  LambdaVision,  a  100-MPixel  wall 
and  (b)  LambdaTable,  a  horizontal  tiled  display  and  interface  (Images 
provided  by  Luc  Renambot,  EVL-UIC  [94,  93]). 

2.4  Projector  Arrays 

Projector  arrays  can  consist  of  CRT,  light- valve,  or  LCD  projectors. 
CRT  projectors  provide  the  most  flexibility  in  terms  of  geometry 
control,  but  have  limited  size  and  brightness  constraints.  The  light- 
valve  projectors  are  bright  and  very  flexible  for  expanding  the  over¬ 
all  size  of  the  array.  Since  several  light- valve  imaging  schemes  are 
driven  by  scanning  CRTs,  they  also  have  good  geometry  controls 
for  the  output  image.  LCD  projectors  offer  a  cost-effective,  low- 
maintenance  solution  for  arrayed  projection;  however,  they  have  vir¬ 
tually  no  geometry  controls. 

Projector  arrays  are  becoming  popular  due  to  their  lack  of  bezels 
and  the  constantly-improving  seamless  integration  of  multiple  tiles. 
A  large  amount  of  research  has  been  done  in  the  area  of  perfect¬ 
ing  both  the  overall  resultant  display  and  the  rendering  algorithms. 
For  example,  research  has  been  performed  on  color  gamut  match¬ 
ing  [129],  seams  [110],  misalignment  [47,  19],  luminance  match¬ 
ing  [71],  and  image  blending  [48].  Projectors  also  offer  what  CRT- 
based,  LCD-based,  and  plasma  monitors  do  not:  a  separation  be¬ 
tween  the  device  size  and  the  size  of  the  displayed  image.  A  small 
projector  can  be  used  to  create  a  very  large  display  or  to  create  a 
very  small  display.  The  possible  range  of  image  size  is  limited  by  lu¬ 
mens,  lens  configurations,  and  available  space.  The  resolution  of  the 
projectors,  continually  improving,  is  also  a  factor  to  consider.  The 
highest  resolution  projection  technology  we  are  aware  of  is  Sony’s 
high  resolution  liquid  crystal  device,  Sony  4K  SXRD  [108],  which 


can  produce  an  image  resolution  of  4096  x  2160.  The  technology 
will  be  used  in  high-end  projectors.  Additionally,  arbitrary  physical 
shapes  can  be  used  as  the  display  surface.  The  result  is  that  pro¬ 
jector  arrays  enable  reconfigurable  and  flexible  display  designs  with 
(theoretically)  little  bezel  distortion  [90,  89]. 

Several  examples  of  commercial  high-resolution  projection-based 
tiled  walls  are  Fakespace’s  PowerWall™  [33],  VisBox’s  stack- 
able,  reconfigurable  projector  array  called  VisBlock™  [124]  (Fig¬ 
ures  3a,  3b  and  Colorplate  Figures  C5),  Cyviz’s  Vizwall™  [24] 
(Figure  4a  and  Colorplate  Figure  C6),  and  Barco’s  control  rooms  [6]. 
Example  installations  are  shown  at  Lawrence  Livermore  National 
Laboratory  [87,  104]  and  UC-Davis  [121].  In  meeting  room  set¬ 
tings,  large  projected  displays  have  been  tiled  enabling  increased  dis¬ 
play  bandwidth  (e.g.,  iLand  [111],  Alias’s  Visualization  Studio  [35], 
DIII-D  National  Fusion  Facility’s  control  room  [1],  and  AT&T’s 
Global  Network  Operations  Center  [130,  20]). 


Figure  3:  Stackable,  reconfigurable  projector  arrays:  (a)  front  and  (b) 
side  views  of  VisBlock™ from  VisBox  Innovative  Display  and  Interac¬ 
tion  Technologies  (Images  courtesy  VisBox.com  [124]). 


2.5  Stereoscopic  Displays 


Figure  4:  Stereoscopic  displays:  (a)  Cyviz’s  stereoscopic  projector 
array  called  Cyviz  Vizwall™ (Image  courtesy  Cyviz.com  [24])  and  (b) 
EVL-UIC’s  autostereoscopic  display  called  Varrier™ (Image  courtesy 
EVL-UIC  [102]). 

Stereoscopic  displays  show  two  sets  of  pixels  for  an  image,  mak¬ 
ing  one  set  visible  to  the  user’s  left  eye  and  the  other  to  the  right 
eye.  Typically  the  user  is  required  to  wear  special  glasses  or  view¬ 
ing  aids  to  see  the  3D  effects.  However,  a  recent  development — 
autostereoscopic  displays — eliminates  the  need  for  special  glasses. 
High-resolution  stereoscopic  displays  are  typically  physically  larger 
than  their  low-resolution  counterparts  and  must  account  for  the  fol¬ 
lowing  factors:  (1)  larger  space  for  head  tracking  (recommended  to 
properly  view  the  3D  effects)  and  (2)  larger  number  of  pixels  to  be 
displayed  (twice  as  many).  These  factors  are  especially  important  if 
the  user  desires  to  interact  in  real-time,  operate  closely  to  the  display 
surface,  and  move  around  the  space  in  front  of  the  display. 

One  of  successful  examples  of  high-resolution  stereoscopic  dis¬ 
plays  is  Cyviz’s  Vizwall  [24]  (Figure  4a  and  Colorplate  Figure  C6). 
It  is  based  on  Cyviz  Viz3D™passive  stereo  capable  display  technol¬ 
ogy  [24],  which  includes  two  projectors  mounted  on  a  positioning 
system.  Vizwall  tiles  several  Viz3D  modules  in  certain  arrays  (3x2 
in  Figure  4a),  leading  to  a  high-resolution  passive  stereoscopic  dis¬ 
play.  EVL-UIC  has  developed  an  autostereoscopic  display  called 
Varrier™  [102]  (Figure  4b  and  Colorplate  Figure  C7),  which  does 
not  require  users  to  wear  any  stereo  glasses  to  view  the  3D  effects. 
Their  approach  involves  a  curved  LCD  tiled  display  with  a  paral¬ 
lax  barrier  affixed  to  the  front.  The  user  is  free  to  move  within  an 


area  of  approximately  32  in  x48  in  (81.3  cm  x  121.9  cm).  In  addi¬ 
tion,  Liao  et  al.  [67]  have  developed  a  high-resolution  display  using 
Integral  Videography  technology  and  9  XGA  projectors  arranged 
in  a  3  x  3  array,  leading  to  a  total  resolution  of  2872  x  2150  pix¬ 
els.  Their  system  generates  geometrically  accurate  high-quality  au¬ 
tostereoscopic  images,  and  reproduces  motion  parallax  in  3D  space 
without  any  special  viewing  glasses  and  head  trackers. 

Another  innovative  high-resolution  stereoscopic  display  is  the  D- 
vision  from  Tokyo  Institute  of  Technology  [46] .  D- vision  uses  24 
projectors  to  provide  stereoscopic  projections  on  a  hybrid  screen.  A 
combination  of  rear  and  front  projection  provides  high-resolution  in 
the  flat  central  region  in  front  of  the  user  and  lower  resolution  on 
curved  screens  around  the  periphery. 


2.6  Volumetric  Displays 

Another  class  of  high-resolution  displays  is  volumetric  displays. 
Rather  than  placing  the  pixels  on  a  single  surface,  they  “stack”  vox¬ 
els  (3D  pixels)  via  different  technologies  to  show  depth.  One  type  of 
volumetric  display  is  the  swept  volume  display,  which  uses  a  fast  ro¬ 
tating  surface  to  which  images  from  one  or  more  projectors  (or  laser 
sources)  are  projected.  The  rotating  display  surface  can  be  opaque 
or  semi-transparent  and  is  rotated  at  sufficient  speeds  to  render  it 
mostly  invisible  to  the  viewer.  Swept  volume  displays  are  available 
or  being  developed  commercially  from  companies  such  as  Actuality 
Systems,  Felix  3D-Display,  Genex  Technologies,  and  Hitachi  [83]. 

Another  type  of  volumetric  display  is  LightSpace  Technologies’ 
DepthCube™  [112]  (Figure  5a,  5b  and  Colorplate  Figure  C8).  The 
DepthCube  uses  a  single  projector  with  twenty  liquid-crystal  projec¬ 
tion  screens  stacked  up  in  front  of  the  projector.  The  screens  are  at 
five-millimeter  intervals.  At  any  given  time,  nineteen  of  the  LCDs 
are  transparent  and  only  one  is  scattering  the  projected  light.  The 
projector  displays  1000  images  per  second  so  the  total  volume  is 
refreshed  at  1000/20  =  50  Hz  refresh  rate. 


(b) 

Figure  5:  Volumetric  displays:  (a)  external  view  and  (b)  cutaway 
mockup  of  DepthCube™ display  from  LightSpace  Technologies  (Im¬ 
age  courtesy  LightSpace  Technologies  [112]) 

The  Xeno Vision  Dynamic  Matrix  Display  [133],  which  may 
also  be  considered  a  volumetric  display,  is  a  table-top  display  that 
moves  the  physical  display  surface  within  a  six-inch  vertical  range 
according  to  an  elevation  file.  Special  rendering  algorithms  cor¬ 
rectly  project  a  full-color  image  file  onto  the  irregular  surface.  The 
3  ft  x4  ft  display  area  takes  about  two  minutes  to  reach  the  desired 
elevations  and  30  seconds  to  erase. 

3  Rendering  and  Streaming 

3.1  Architectures  and  Data  Distribution 

Cluster  rendering  can  be  described  as  the  use  of  a  set  of  computers 
connected  via  a  network  for  rendering  purposes.  Rendering  may 
take  many  forms:  distributed  video  streaming,  non-photorealistic 
volume  rendering,  ray  tracing  and  radiosity-based  rendering,  or  in¬ 
teractive  rendering  using  application  programming  interfaces  (APIs) 
like  OpenGL.  We  distinguish  between  display  data  streaming  soft¬ 
ware  and  distributed  rendering  software.  Data  streaming  toolkits 
enable  streaming  of  any  type  of  data  for  large  display  systems. 


Molnar  et  al.  [74]  classified  parallel  3D  rendering  algorithms  into 
three  general  classes  based  on  when  the  sorting  of  the  primitives 
occurs  in  the  transition  from  object  to  screen  space.  The  three  classes 
are  sort-first,  sort-middle,  and  sort-last. 

Sort-first  In  sort-first  algorithms,  the  display  is  partitioned  into  dis¬ 
crete,  disjoint  tiles.  Each  rendering  node  of  the  cluster  is  then  as¬ 
signed  one  or  more  of  these  tiles  and  is  responsible  for  the  complete 
rendering  of  only  those  primitives  that  lie  within  one  of  its  tiles. 

Sort-middle  Sort-middle  algorithms  begin  by  distributing  each 
graphics  primitive  to  exactly  one  “processor”  for  geometry  process¬ 
ing.  After  the  primitive  has  been  transformed  into  screen  space,  it  is 
forwarded  to  another  processor  for  rendering. 

Sort-last  In  sort-last  approaches,  each  primitive  is  sent  to  exactly 
one  node  for  rendering.  After  all  primitives  have  been  rendered,  the 
nodes  must  composite  the  images  to  form  the  final  image.  This  usu¬ 
ally  requires  a  large  amount  of  bandwidth  because  each  node  must 
send  the  entire  image  to  a  compositor. 

This  taxonomy  is  generally  conceived  as  considering  a  single,  multi¬ 
processor  machine,  but  it  applies  equally  well  to  a  cluster  of  inde¬ 
pendent  machines  that  produce  a  single  conceptual  display.  Tiled 
displays  lead  naturally  toward  a  sort-first  approach.  The  screen  is 
already  partitioned  into  tiles,  with  each  tile  driven  by  a  single  cluster 
node.  Having  multiple  machines  with  separate  memories  compli¬ 
cates  the  task  of  keeping  the  data  consistent,  however. 

Cluster  rendering  systems  vary  widely  with  respect  to  the  way 
data  is  distributed  among  the  cluster  nodes.  Chen  et  al.  [17,  18]  first 
looked  at  the  problem  of  data  distribution.  Two  general  models  have 
emerged:  client-server  and  master-slave. 

Client-server  In  the  client-server  model,  a  user  interacts  with  a  sin¬ 
gle  instance  of  the  application  that  runs  on  a  client  node.  This  client 
generates  the  geometry  and  distributes  it  to  the  render  servers  (Fig¬ 
ure  6a).  Graphics  processors  offer  two  rendering  modes  -  immediate 
mode  and  retained  mode.  In  immediate  mode,  the  client  sends  the 
primitives  over  the  network  every  frame.  In  retained  mode,  each 
render  server  stores  and  reuses  primitives  it  receives.  The  client  then 
needs  to  send  only  changes  to  the  geometry.  This  method  is  usually 
accomplished  through  a  scene  graph. 

Master-slave  In  the  master-slave  model,  the  application  executes 
on  every  cluster  node.  Execution  of  the  application  on  all  nodes 
must  be  synchronized  to  ensure  consistency  among  all  application 
instances.  Typically,  a  master  node  handles  all  user  interaction  and 
synchronizes  state  changes  between  all  other  nodes  (Figure  6b). 


Figure  6:  Cluster  rendering  distribution  schemes:  (a)  client-server, 
(b)  master-slave. 

The  master-slave  approach  usually  requires  the  least  amount  of 
bandwidth.  The  results  of  user  interactions  and  other  state  changes 
are  sporadic  and  relatively  simple  to  transmit  over  a  network.  This 
approach,  however,  is  not  transparent,  as  everything  affecting  pro¬ 
gram  execution  must  be  considered  as  input.  Timers,  random  num¬ 
ber  generation,  system  calls,  or  any  variables  influencing  program 
execution  need  to  be  distributed  and  synchronized  among  the  nodes. 
The  client-server  approach  is  usually  transparent  to  the  programmer. 
The  program  can  be  implemented  as  if  it  were  running  on  a  single 
machine  and  the  system  will  handle  the  rest. 


3.2  Display  Data  Streaming  Software 

There  are  several  software  toolkits  that  enable  streaming,  splitting, 
and  displaying  many  types  of  data  (e.g.,  2D  imagery,  videos,  and 
desktop  content)  for  large  display  systems. 

TeraVision  The  TeraVision  system  captures  and  distributes  vi¬ 
sual  imagery  from  any  graphics  platform  over  a  high-speed  net¬ 
work  [107].  TeraVision  has  been  demonstrated  to  successfully 
stream  real-time  microscopy  images  at  2000  x  2000  pixel  resolu¬ 
tion  from  the  National  Center  for  Microscopy  and  Imaging  Research 
(NCMIR)  to  Hawaii,  to  provide  researchers  at  the  University  of 
Hawaii  access  to  NCMIR’ s  microscope. 

SAGE  EVL-UIC  has  developed  the  Scalable  Adaptive  Graphics  En¬ 
vironment  (SAGE),  a  graphics  streaming  architecture  that  supports 
seamless  display  of  networked  applications  across  high-resolution 
displays.  The  data  from  the  applications  may  include  3D  render¬ 
ings,  video  streams,  2D  geospatial  imagery,  etc.  SAGE  supports 
hundreds  of  megapixels  of  contiguous  display  resolution  and  pro¬ 
vides  dynamic  pixel  routing  capability,  which  allows  users  to  freely 
move  and  resize  an  application’s  imagery  over  tiled  displays  in  real 
time  [56,  96].  SAGE  relies  on  TeraVision  to  distribute  visual  im¬ 
agery  across  different  platforms  [107]. 

EVL-UIC  VNC  Viewer  EVL-UIC  developed  a  Virtual  Network 
Computer  (VNC)  protocol  client  that  enables  users  to  bring  desk¬ 
top  content  to  the  SAGE  environment.  EVL-UIC’s  VNC  Viewer  is  a 
standard  VNC  client  program  modified  to  serve  as  a  proxy  between 
a  VNC  server  (of  any  size  and  pixel  depth)  and  SAGE.  Once  the 
pixels  are  retrieved  from  the  VNC  server,  the  same  pixels  are  given 
to  the  SAGE  API  for  immediate  display.  SAGE  supports  any  num¬ 
ber  of  simultaneous  VNC  applications,  making  use  of  the  large  real 
estate  offered  by  high-resolution  tiled  displays. 

IBM’s  Scalable  Graphics  Engine  Perrine  and  Jones  [85]  developed 
a  parallel  rendering  environment  for  the  IBM  Scalable  Graphics  En¬ 
gine  (SGE).  Their  toolkit  supports  tunneling,  which  allows  graphics 
applications  to  communicate  directly  with  the  SGE,  SMP  render¬ 
ing,  and  includes  an  OpenGL  implementation  that  utilizes  the  SGE. 
Based  on  this  toolkit,  they  implemented  a  parallel  MPEG  video 
player. 

VLC  Media  Player  The  VLC  media  player  [125]  is  a  cross-platform 
video  player  that  supports  a  large  number  of  video  and  audio  codecs. 
VLC  support  media  streaming  and  includes  a  “wall”  video  filter  that 
splits  the  output  in  several  tiles. 

OptiStore  and  LambdaRAM  OptiStore  is  a  high-performance, 
low-latency  data  retrieval  system  that  filters  raw  2D  and  3D  volumet¬ 
ric  data  and  produces  a  sequence  of  visual  objects.  OptiStore  uses 
LambdaRAM  to  achieve  low-latency  data  access  by  aggressively  us¬ 
ing  high  bandwidth  networks  and  caching. 

JuxtaView  JuxtaView  is  an  application  for  visualizing  extremely 
high-resolution  montage  images  on  scalable  tiled  displays,  which  are 
able  to  deal  with  the  largest  montages  produced  by  biologists  [61]. 
JuxtaView  uses  OptiStore  to  enable  rapid  panning  and  zooming 
through  enormous  images  accessed  over  wide-area  high-speed  net¬ 
works. 

Scalable  Visualization  Consumer  The  Scalable  Visualization 
Consumer  (SVC)  receives  MPEG2  data  through  a  Firewire® 
(IEEE  1394)  interface,  files  on  disk,  or  network  interface  and  de¬ 
compresses  it  for  streaming  to  a  tiled  display.  The  MPEG2  data  is 
decompressed,  split  as  sub-images,  and  streamed  to  the  appropriate 
display  nodes. 


NCSA  Pixel  Blaster  The  National  Center  for  Supercomputing  Ap¬ 
plications  (NCSA)  developed  Pixel  Blaster  [79],  a  distributed  high- 
definition  movie  player,  which  reads  raw-format  HD  images  and  dis¬ 
tributes  them  to  display  nodes.  The  application  was  ported  by  EVL- 
UIC  to  operate  with  SAGE. 

3.3  Distributed  Rendering  Software 

Raffin  and  Soares  [88]  present  a  review  of  common  software  toolk¬ 
its  supporting  PC  clusters  for  Virtual  Reality  systems,  including 
CAVELib,  VR  Juggler,  Syzygy,  OpenSG,  Chromium,  and  others. 
Although  they  discuss  distributed  rendering  software  in  the  context 
of  VR  and  parallelism,  these  tools  are  naturally  applicable  to  large 
high-resolution  display  systems.  For  example,  Chromium  [50]  has 
been  widely  used  to  support  interactive  parallel  visualization  appli¬ 
cations  displaying  to  tiled  displays.  OpenSG  [127]  implements  ren¬ 
dering  on  tiled  displays  by  dividing  the  screen  into  MxN  uniformly 
spaced  tiles  with  a  render  server  assigned  to  each  of  them.  We  sum¬ 
marize  several  additional  toolkits  that  specifically  handle  rendering 
for  large  high-resolution  display  systems. 

Distributed  Multihead  X  (DMX)  Typical  X- Windows  servers  pro¬ 
vide  support  for  multiple  displays  connected  to  the  same  machine 
via  the  Xinerama  extension.  The  DMX  project  [29]  provides  a  proxy 
X  server  that  is  a  front  end  to  X  servers  running  on  each  rendering 
node  in  a  cluster.  The  X  client  application  will  connect  to  the  front- 
end  server;  rendering  requests  will  be  broken  down  as  needed  and 
sent  to  the  appropriate  back-end  servers  via  XI 1  library  calls.  DMX 
is  transparent  to  the  application  and  supports  standard  mouse  and 
keyboard  input  through  the  XInput  extension. 

Aura  Aura  [122]  is  a  multi-platform  API  designed  for  scientific  vi¬ 
sualization  on  tiled  displays.  In  “broadcast”  mode  it  implements  a 
client-server  model.  It  provides  the  user  with  a  scene  graph  interface 
to  take  advantage  of  frame-to-frame  coherence.  Aura  also  provides 
a  master-slave  configuration  called  “multiple  copies”. 

Virtual  Immersive  Reality  Program  Interface  (VIRPI)  Ger¬ 
mans  et  al.  [40]  developed  VIRPI  on  top  of  Aura  [122]  to  provide  a 
high-level  user  interface  toolkit.  VIRPI  provides  standard  widgets, 
such  as  menus  and  radio  buttons,  as  well  as  a  framework  for  mea¬ 
suring  in  virtual  spaces. 

Blue-c  Distributed  Scene  Graph  The  blue-c™  distributed  scene 
graph  by  Naf  et  al.  [78]  is  based  on  OpenGL  Performer  [100].  To 
support  collaboration  and  cluster  rendering,  it  has  been  enhanced 
with  node  serialization  and  state  update  interfaces.  Additional  cus¬ 
tom  nodes  and  attribute  objects  are  integrated  to  support  multimedia 
elements  [76,  77]. 

Virtual  Graphics  Platform  (VGP)  ModViz,  Inc.  developed  the  Vir¬ 
tual  Graphics  Platform  [123],  a  cluster  rendering  toolkit  for  Linux 
environments.  Similar  to  Chromium,  it  is  transparent  to  the  appli¬ 
cation  and  intercepts  OpenGL  function  calls,  which  are  distributed 
to  rendering  servers  running  on  each  cluster  node.  VGP  supports 
large-scale  multi-screen  projection  displays  and  image  compositing. 

Renderizer  ModViz,  Inc.’s  Renderizer  [123]  enables  multi-display 
cluster  rendering  for  OpenGL  Performer-based  applications.  It  re¬ 
places  a  small  number  of  OpenGL  Performer  function  calls  and 
eliminates  the  need  to  parallelize  existing  applications  explicitly. 

EVL-UIC  OpenGL  Wrapper  The  EVL-UIC  OpenGL  Wrapper  al¬ 
lows  easy  porting  of  native  OpenGL  applications  to  SAGE.  The 
wrapper  operates  similarly  to  WireGL  by  intercepting  the  “glSwap- 
Buffer”  command,  and  then  it  streams  the  OpenGL  data  to  SAGE. 

Vol-a-Tile  Vol-a-Tile  [106]  is  a  volume  visualization  tool  for  large- 
scale,  time- series  scientific  datasets  rendered  on  high-resolution 


scalable  displays.  These  large-scale  datasets  can  be  dynamically 
processed  and  retrieved  from  remote  data  stores  over  optical  net¬ 
works  using  OptiStore.  Vol-a-Tile  utilizes  the  fast  OpenGL  3D  tex¬ 
turing  and  fragment  shaders. 

NCSA  TerraServer  Blaster  The  NCSA  TerraServer 
Blaster  [80]  is  an  OpenGL-based  interface  to  the  terraserver 
(www.terraservice.net),  a  3. 3 -terabyte  online  database  of  high 
resolution  USGS  aerial  imagery  for  all  of  the  United  States.  It 
can  be  used  together  with  Chromium  to  display  on  high-resolution 
display  walls. 

4  Applications  of  Large  High-Resolution  Displays 
4.1  Command  and  Control 

Large  high-resolution  displays  have  been  widely  installed  in  com¬ 
mand  and  control  centers  for  a  variety  of  applications  including 
military,  aerospace,  and  telecommunications.  The  Air  Force  Re¬ 
search  Laboratory  developed  the  Interactive  DataWall,  which  is  an 
ultra-high-resolution  large  screen  display  that  has  an  interface  us¬ 
ing  wireless  interaction  devices.  Jedrysik  et  al.  [55]  use  the  Interac¬ 
tive  DataWall  for  situational  awareness  and  collaborative  decision¬ 
making  tasks  involving  battlefield  data. 

A  large  four- wall  immersive  room  has  been  deployed  at  the  Naval 
Research  Laboratory  (NRL).  The  3D  Virtual  &  Mixed  Environment 
Laboratory  developed  a  submarine  command  application  for  detect¬ 
ing  target  submarines  that  operates  in  the  immersive  room  [105].  A 
sonar  operator  is  immersed  in  an  oceanographic  view,  where  a  set 
of  tracking  sonar  buoys  are  dropped,  and  can  visualize  the  output  of 
several  different  tracking  algorithms  that  have  been  developed  (Fig¬ 
ure  7a  and  Colorplate  Figure  C9). 

Christie  constructed  a  large,  high-resolution,  198,000  ft2  commu¬ 
nication  command  and  control  center  for  AT&T  in  1999  [130,  20]. 
The  control  center,  AT&T’s  Global  Network  Operations  Center,  has 
given  AT&T  the  unparalleled  capability  to  manage  the  flow  of  com¬ 
munications  traffic  across  its  network  anywhere  in  the  world  from 
one  location.  The  center  contains  over  75  high-resolution  projected 
displays,  each  used  to  visualize  computer-generated  data  and  graph¬ 
ics  associated  with  over  250  million  voice  calls  on  a  typical  weekday. 


Figure  7:  Applications:  (a)  NRL’s  immersive  room  demonstrating  a 
submarine  command  &  control  application,  and  (b)  Deere  &  Com¬ 
pany’s  VR  testing  facility  for  virtual  prototyping  vehicle  designs  and 
testing  their  effectiveness  (Photo  courtesy  Deere  &  Company,  Moline, 
Illinois  [27]) 

4.2  Vehicle  Design 

It  has  been  a  fundamental  requirement  of  the  automotive  design  in¬ 
dustry  to  display  and  interact  with  vehicle  models  at  1:1  scale  [15]. 
Therefore,  automotive  design  studios  have  explored  the  use  of  a  vari¬ 
ety  of  large-format  digital  displays  in  their  design  workflow,  includ¬ 
ing  tape  drawing,  electronic  drafting  tables,  ImmersaDesk™VR 
systems,  CAVEs,  and  PowerWalls.  Deere  &  Company  is  develop¬ 
ing  applications  that  simulate  vehicle  operations  using  large  VR  dis¬ 
play  facilities  like  the  one  shown  in  Figure  7b  and  Colorplate  Fig- 


ure  CIO.  They  utilize  such  applications  to  evaluate  human  factors 
and  ergonomics,  analyze  complex  engineering  data,  and  build  capa¬ 
bilities  in  vehicle  manufacturing  process  development  [27]. 

4.3  Geospatial  Imagery  and  Video 

Large  high-resolution  displays  offer  the  sense  of  scale  needed  for 
geospatial  imaging  and  large  film-quality  video  applications.  The 
ability  to  obtain  realistic  terrain  representations,  zoom  across  scales, 
and  create  fly-through  animations  certainly  benefits  geospatial  visu¬ 
alization.  Large  displays  allow  users  to  see  critical  details  in  com¬ 
plex  dynamic  phenomena,  such  as  subtle  eddies  that  are  critical  to 
understanding  global  ocean  circulation  models  [22]. 

High-resolution  display  systems  are  used  by  several  major  oil  and 
gas  companies  for  geospatial  exploration  and  engineering,  3D  map¬ 
ping,  and  geophysical  analysis  [31].  Evans  et  al.  [32]  used  high- 
resolution  displays  to  develop  interactive  prototypical  spatial  mod¬ 
els  of  forest  stands  from  LIDAR  and  multi- spectral  data.  They  use 
the  large  displays  to  visualize  a  spatially  true  models  of  the  stands. 
MacEachren  [69]  studied  how  the  the  large  displays  impact  geospa¬ 
tial  analysis.  They  revealed  that  iconic  data  representations,  interac¬ 
tion  methods,  and  remote  collaboration  techniques  need  to  be  con¬ 
sidered  when  using  the  displays  for  geospatial  analysis. 

4.4  Scientific  Visualization 

Large  high-resolution  displays  have  been  one  of  the  favorite  choices 
for  scientific  visualization  applications  because  they  offer  (1)  view¬ 
ing  of  data  at  true-to-life  or  human- scale  physical  sizes  and  (2)  view¬ 
ing  of  large  amounts  of  data  simultaneously  with  the  increased  num¬ 
ber  of  pixels  available.  One  organization  using  large  high-resolution 
display  for  scientific  visualization  is  Oak  Ridge  National  Laboratory. 
They  are  performing  a  variety  of  scientific  visualization  research  in¬ 
cluding  scientific  simulations  of  galactic  supemovae,  protein  expres¬ 
sion,  nanostructure  models,  and  fusion  energy  devices  [81].  They 
have  a  30  ft  x8  ft  display  wall  with  a  resolution  of  more  than 
11,000  x  3,000  pixels  (35  million  pixels)  installed  at  their  Science 
Visualization  Facility.  Another  organization  is  EVL-UIC,  which  is 
developing  applications  for  visualizing  geoscience  data  like  seismic 
activity,  flow  of  water  through  the  earth,  and  physical  processes  of 
the  inner  earth.  The  applications  have  been  designed  for  their  high- 
resolution  tile  wall  called  GeoWall2  (a  5  x  3  array  of  LCD  panels) 
and  other  displays  [66]. 

4.5  Collaboration  and  Tele-immersion 

An  integral  part  of  collaborative  work  is  a  public  display  surface  that 
serves  as  a  medium  for  presenting,  capturing,  and  exchanging  ideas. 
Several  examples  of  display  surfaces  and  collaborative  research  as¬ 
sociated  with  them  are  itemized. 

Liveboard  Xerox  PARC’s  Elrod  et  al.  [30]  pioneered  a  large  inter¬ 
active  display  system  called  Liveboard,  which  provides  computer- 
supported  group  meetings,  presentations,  and  remote  collaboration. 
The  Liveboard  is  a  stylus-based,  interactive  large-format  display, 
which  can  be  used  as  a  meeting- support  tool  providing  whiteboard¬ 
like  functionality,  allowing  users  to  write  down  ideas  and  retrieve 
documents. 

UNC’s  Office  of  the  Future  UNC  [91]  proposed  the  technology 
of  “Office  of  the  Future”,  combining  wall-sized  high-resolution  dis¬ 
plays,  cameras,  and  several  types  of  interaction  techniques.  The  goal 
of  the  system  is  to  provide  very  compelling  tele-collaboration  among 
distant  individuals. 

Dynamo  Izadi  et  al.  [54]  designed  and  implemented  Dynamo,  a 
communal  multi-user  interactive  surface  with  which  users  can  eas¬ 
ily  access  and  interact.  This  device  allows  people  meeting  in  public 


spaces  to  share  and  manipulate  digital  information,  such  as  docu¬ 
ments,  video,  and  images,  in  a  manner  that  emulates  physical  docu¬ 
ments. 

Alias  Visualization  Studio  The  Alias  Visualization  Studio  [35,  2] 
houses  a  number  of  state-of-the-art  corporate  meeting  facilities,  in¬ 
cluding  large  display  walls  (Colorplate  Figure  Cll).  The  designers 
indicated  the  importance  of  good  interior  design,  drawing  on  theatri¬ 
cal  and  cinematic  principles,  and  social  concerns,  such  as  privacy, 
aesthetics  of  large  displays,  and  attention  management  (including 
distractions  on  secondary  displays). 

DIII-D  National  Fusion  Facility  Control  Room  The  DIII-D  Na¬ 
tional  Fusion  Facility  control  room  [1]  has  installed  a  large,  shared 
display  wall  composed  of  three  50-inch  Toshiba  P500DK  data  wall 
cubes  arranged  horizontally  creating  a  nearly  seamless  3840  x  1240 
pixel  tiled  display.  The  US  National  Fusion  Collaboratory  Project 
is  utilizing  the  large  shared  display  wall  for  control  room  collabo¬ 
rations  in  the  following  ways:  (1)  presenting  up-to-date  information 
about  experiment  status  and  group  activity;  (2)  sharing  data  between 
personal  desktop  screens  and  the  large  display  wall;  and  3)  video- 
conferencing  of  core  team  members  from  remote  sites. 

Tele-immersion  is  a  new  paradigm  of  collaboration.  It  holds  a 
promise  of  creating  collaborative  visualization  and  VR  applica¬ 
tions  over  national  and  global  high-speed  networks.  A  large  high- 
resolution  display  is  an  ideal  facility  for  tele-immersion  applica¬ 
tions,  since  collaborative  exploration  of  massive  scientific  data  sets 
requires  a  large  screen  real  estate.  In  the  late  1990s,  the  National 
Tele-immersion  Initiative  (including  UNC-Chapel  Hill,  Brown,  and 
Univ.  of  Pennsylvania)  was  formed  to  experiment  with  realistic  hu¬ 
man  representation  at  a  distance  (over  Intemet2)  in  order  to  facil¬ 
itate  tele-collaboration  [101].  The  blue-c  project  expanded  those 
results  and  unified  them  with  lessons  learned  from  CAVE-related 
research  [41,  64,  77]. 

The  Air  Force  Office  of  Scientific  Research  is  supporting  work 
at  Iowa  State  University  to  develop  new  technology  for  monitoring 
and  control  of  unmanned  aerial  vehicles  (UAVs).  Iowa  State  is  de¬ 
veloping  a  system  that  provides  real-time  data  in  a  tele-immersive 
large-scale  display  environment  and  allows  operators  better  control 
of  the  UAVs  [128]. 

4.6  Education  and  Training 

Large  high-resolution  displays  are  a  great  tool  for  education  and 
training  in  astronomy,  bioinformatics,  medical  imaging,  urban  plan¬ 
ning,  and  geographic  information.  Researchers  at  EVL-UIC  have 
built  an  advanced  tiled- visualization  system  called  GeoWall2  to  de¬ 
velop  informal  science  education  applications  for  museums,  gal¬ 
leries,  and  other  public  events  [66].  UC-Santa  Cruz  [73]  has  de¬ 
veloped  a  collaborative  learning  environment  for  the  classroom  by 
using  a  large  shared  tile- wall  display.  The  display  space  is  shared  by 
the  instructor  and  students.  The  large  display  space  provides  the  pri¬ 
mary  means  of  presentation  of  lecture  material,  allowing  the  lecturer 
to  keep  multiple  screens  of  material  in  view  for  the  students.  The 
students  can  download  any  portion  of  the  display,  enabling  them  to 
go  back  at  any  time  to  previous  screens  of  the  presentation.  The  dis¬ 
play  also  permits  students  to  show  their  work,  ideas,  and  even  pose 
questions. 

4.7  Immersive  Applications 

Iowa  State  University’s  Virtual  Reality  Applications  Centers 
(VRAC)  has  been  developing  immersive  applications  with  their 
large-display  environments  over  the  last  several  years  [128].  They 
have  projects  for  virtual  prototyping  thermal  systems  in  a  power 
plant  and  developing  virtual  engineering  tools  for  livestock  produc¬ 
tion.  The  Naval  Research  Laboratory  [119]  and  others  [128]  have 


developed  fire  training  tools  using  an  immersive  room  that  enable 
users  to  practice  fire  safety  procedures  inside  of  buildings,  ships, 
and  other  environments. 


4.8  Public  Information  Displays 

Advances  in  flat  panel  display  technology  and  projector  research 
have  led  to  consumer  grade  devices  and  the  ability  to  use  them  in 
many  novel  ways,  such  as  the  use  of  high-resolution  imagery  in  per¬ 
vasive  public  displays.  Where  there  once  might  have  been  a  printed 
image  or  a  low  grade  digital  image,  we  are  now  starting  to  see  large 
tiled  public  displays  [36].  No  longer  are  the  displays  limited  to  flat 
billboards  either — most  any  surface  or  collection  of  surfaces  has  the 
potential  to  receive  digital  enhancement  [28]. 

Many  of  the  concepts  of  pervasive  displays  have  roots  in  ambient 
spaces  research  [132].  In  traditional  computer  applications,  includ¬ 
ing  YR,  user  actions  are  explicit.  That  is,  users  consciously  direct 
the  computer  to  perform  some  desired  task.  In  ambient  and  per¬ 
vasive  applications,  actions  could  be  implicit.  The  computer  could 
recognize  an  action  and  respond  appropriately,  even  when  there  is 
no  intended  computer  interaction  or  an  awareness  that  user  actions 
could  trigger  a  response  from  the  computer.  Inherent  in  this  effort  to 
separate  the  display  from  the  computing  hardware  are  many  of  the 
ideals  expressed  in  ubiquitous  computing  [131]. 


5  Visual  Effects  and  Human  Performance 

When  large  interactive  displays  were  constructed,  researchers  real¬ 
ized  that  the  changing  visual  effects  afforded  by  the  increased  dis¬ 
play  landscape  would  have  a  profound  impact  on  how  users  work 
with  computing  workspaces.  As  Swaminathan  and  Sato  [113]  ob¬ 
served,  “when  a  display  exceeds  a  certain  size,  it  becomes  qualita¬ 
tively  different.”  Prior  research  not  only  qualitatively  observes  in  de¬ 
tail  user  behavior  on  such  displays,  but  also  quantitatively  explores 
visual  effects  of  large  high-resolution  displays  on  users’  task  perfor¬ 
mance.  Practical  usability  issues  of  both  hardware  and  software  have 
been  uncovered,  and  guidelines  for  design  and  presentation  of  infor¬ 
mation  systems  on  large  high-resolution  displays  have  been  formed. 


5.1  Qualitative  Evaluations 

Large  displays  were  initially  applied  in  building  collaborative 
workspaces,  and  benefits  of  large-format  displays  for  groups  work¬ 
ing  together  have  been  demonstrated.  Despite  the  Liveboard 
project’s  focus  on  underlying  hardware  and  a  pen-based  user  in¬ 
terface,  an  informal  survey  found  it  was  most  frequently  used  for 
group  meeting  facilitation.  Guimbretiere  et  al.  [44]  also  explored 
pen-based  interaction  with  high-resolution  wall-size  displays,  and 
have  tested  their  design  with  professional  product  design  groups  en¬ 
gaged  in  brainstorming  tasks. 

Ball  et  al.  [4]  reported  an  observational  analysis  of  the  use  of 
a  large  tiled  display  consisting  of  9  LCD  monitors  in  3  x  3  array 
over  the  course  of  six  months.  Although  it  is  not  a  controlled  ex¬ 
perimental  evaluation  in  its  nature,  it  provides  insightful  feedback 
concerning  common  usage  of  how  users  do  and  do  not  use  a  large 
high-resolution  display  to  perform  ordinary  tasks,  such  as  reading 
papers,  surfing  web  pages,  viewing  images,  programming,  and  en¬ 
tertaining.  For  example,  they  found  a  bezel  adaptation  strategy  em¬ 
ployed  by  most  users  when  working  with  tiled  LCD  surfaces,  show¬ 
ing  that  bezels  tend  to  help  users  quickly  separate  multiple  appli¬ 
cations  and  tasks,  which  significantly  decreases  context  switching. 
Based  on  their  observations,  they  summarized  the  advantages  and 
disadvantages  of  using  tiled  high-resolution  displays  and  formed  de¬ 
sign  recommendations  and  guidelines  for  application  designers. 


5.2  Quantitative  Tests  of  Spatial  Tasks 

With  LCD  panels  becoming  less  expensive,  it  is  an  emerging  trend 
to  have  personal  multi-monitor  configurations  in  offices.  Also,  it  is 
not  uncommon  to  install  individual  large  displays  with  affordable 
projectors.  It  therefore  opens  up  numerous  research  opportunities 
with  respect  to  individual  gains  and  user  behavior  on  large  displays. 

Tan  et  al.  [117,  116]  conducted  two  studies  to  quantify  benefits 
of  physically  large  displays  for  individual  users.  They  took  an  ap¬ 
proach  of  designing  controlled  experiments,  identifying  independent 
and  dependent  variables  explicitly  while  holding  constant  other  fac¬ 
tors.  This  approach,  compared  to  previous  practical  experiments,  al¬ 
lows  precise  statistical  analysis  of  the  results  to  identify  the  causes  of 
any  observed  difference  in  performance  or  usability  data  [14].  They 
maintained  a  constant  viewing  angle  for  each  of  two  displays  in  their 
experimental  design.  In  the  first  study,  participants  performed  spa¬ 
tial  orientation  tasks  involving  static  2D  scenes.  A  significant  per¬ 
formance  gain  was  observed  on  the  large  display,  even  though  the 
two  displays  cast  identical- size  retinal  images.  In  the  second  study, 
they  designed  triangle  completion  tasks  to  examine  how  display  size 
affected  path  integration  performance  in  interactive  3D  virtual  envi¬ 
ronments.  Not  surprisingly,  users  were  more  effective  on  the  physi¬ 
cally  large  display.  They  suggested  that  large  displays  may  afford  a 
greater  sense  of  presence,  leading  users  to  adopt  an  egocentric  rather 
than  an  exocentric  strategy  in  spatial  tasks.  Further  studies  [117] 
found  the  benefits  of  being  able  to  interact  with  the  environment 
were  independent  of  the  physical  size  of  the  display. 

Researchers  have  also  been  interested  in  examining  how  large  dis¬ 
plays  reduce  gender  bias  in  navigating  virtual  environments.  Exist¬ 
ing  reports  have  suggested  that  males  significantly  outperformed  fe¬ 
males  in  YE  navigation.  However,  Czerwinski  et  al.  [26]  undertook 
controlled  experiments,  uncovering  that  women  were  able  to  achieve 
similar  VE  navigation  performance  to  men  if  they  were  exposed  to 
a  large  display  coupled  with  a  wide  field  of  view.  A  follow-up  study 
by  Tan  et  al.  [115]  attempted  to  identify  what  factors  were  driving 
3D  navigation  gains  for  females.  They  indicated  that  the  gender- 
specific  navigation  benefits  come  from  the  presence  of  optical  flow 
cues,  which  were  better  afforded  by  a  wider  field  of  view  on  large 
displays. 

While  recommendations  and  guidelines  for  design  and  presenta¬ 
tion  of  interactive  3D  virtual  environments  on  large  displays  have 
been  made,  we  lack  insights  in  the  effectiveness  of  large  high- 
resolution  displays  for  basic  low-level  data  visualization  and  naviga¬ 
tion  tasks.  Ball  et  al.  [5]  described  an  exploratory  study  comparing 
basic  visualization  task  performance  on  a  large  tiled  display  with  a 
resolution  of  3840  x  3072  and  two  smaller  displays  (1560  x  2048 
and  1280  x  1024).  They  concluded  that  with  finely  detailed  data, 
large  high-resolution  displays  result  in  more  physical  navigation, 
which  is  preferable  to  virtual  zoom-and-pan  navigation  on  smaller 
displays. 

In  addition,  researchers  have  started  to  explore  the  possible  us¬ 
age  of  large-format  displays  in  creating  semi-immersive  environ¬ 
ments.  Patrick  et  al.  [84]  conducted  an  empirical  study  to  examine 
how  users  acquired  spatial  knowledge  in  a  VE  presented  on  a  head- 
mounted  display  (HMD),  a  large  projection  display,  and  a  desktop 
monitor.  An  intriguing  finding  was  there  was  no  observed  signifi¬ 
cant  difference  in  reproduction  of  VE- survey  knowledge  when  users 
viewed  through  an  HMD  versus  a  large  screen,  implying  that  large 
displays  may  be  an  effective  low-cost  alternative  for  HMDs  for  VE 
applications. 

5.3  Quantitative  Tests  for  Information  Display 

Polys  et  al.  [86]  considered  how  varied  display  size  affects  informa¬ 
tion  layout  interface  design  in  an  information-rich  virtual  environ¬ 
ment  (IRVE).  Two  information  layout  techniques  to  support  search 
and  comparison  tasks  were  designed  and  evaluated.  The  Object 


Space  technique  was  to  associate  textual  labels  relative  to  their  ref¬ 
erent  virtual  objects,  and  the  Viewport  Space  technique  was  to  dis¬ 
play  labels  on  an  image  plane  workspace.  They  altered  two  levels 
of  software  field  of  view  (SFOV)  on  a  single  monitor  and  a  3  x  3 
tiled  LCD  display,  respectively.  Search  and  comparison  tasks  were 
designed,  and  both  accuracy  and  time  were  measured.  They  iden¬ 
tified  a  significant  advantage  provided  by  the  Viewport  Space  tech¬ 
nique  combined  with  a  wide  SFOV.  Since  an  IRVE  is  a  spatial,  per¬ 
ceptual  VE  enriched  by  abstract  information,  it  is  useful  in  many 
application  domains  where  pure  perceptual  information  is  not  suffi¬ 
cient,  such  as  scientific  visualization,  military  simulation,  architec¬ 
ture  walkthrough,  education,  and  entertainment.  Intuitively,  IRVE 
applications  may  benefit  greatly  from  large  high-resolution  displays. 
Polys  et  al.’s  work  represents  a  trend  toward  designing  and  evaluat¬ 
ing  original  techniques  for  new  displays  to  increase  users  productiv¬ 
ity  in  IRVE  applications. 

The  Infocockpit  [118]  showed  that  the  memory  retention  of  a 
user  was  improved  56%  by  using  multiple  monitors  to  spread  the 
information  around  the  user’s  space,  creating  a  sense  of  presence. 
Lin  et  al.  [68]  found  a  correlation  between  memory  retention  of  a 
virtual  environment,  the  sense  of  presence  gained  from  within  that 
environment,  and  the  size  of  the  field  of  view  (FOV).  They  showed 
that  increased  FOV  improved  performance  on  memory-based  tasks 
and  correlated  with  a  better  sense  of  presense.  They  did  not  attempt 
to  establish  evidence  for  the  cause  of  this  correlation. 

6  User  Interfaces  and  Interaction  Techniques 

While  user  interfaces  for  standard  desktop  displays  have  been  devel¬ 
oped  over  a  few  decades  now,  there  has  been  relatively  little  work 
on  interfaces  for  large  format  displays.  We  identify  large  display  us¬ 
ability  issues  and  interaction  challenges,  and  then  examine  the  work 
in  the  field  by  dividing  the  efforts  into  two  categories:  work  aimed 
at  2D  displays  and  work  aimed  at  3D  displays.  While  techniques  for 
the  former  tend  to  emerge  as  extensions  of  desktop  interfaces,  tech¬ 
niques  for  the  latter  share  more  with  virtual  reality  (VR)  interfaces. 

6.1  Usability  Issues  and  Interaction  Challenges 

To  design  usable  and  useful  interface  and  interaction  for  large  high- 
resolution  displays,  we  must  understand  which  factors  have  left  con¬ 
ventional  interface  techniques  awkward  to  use  on  large  displays.  We 
identify  five  categories  of  large  display  usability  issues. 

1.  Reaching  distant  objects.  As  screen  real  estate  grows,  it  is  in¬ 
creasingly  difficult  for  users  to  access  objects  scattered  around  on  a 
wall-sized  display  [98],  especially  when  they  tend  to  stay  relatively 
close  to  a  large  display.  For  example,  if  a  user  seated  in  front  of 
a  large  screen  tries  to  drag  a  file  icon  near  the  lower  right  comer 
to  the  recycle  bin  icon  on  the  left,  it  will  be  a  terrible  experience 
to  use  a  traditional  drag-and-drop  interaction  paradigm.  It  is  also  a 
common  problem  with  more  modest  multi-monitor  configurations, 
since  accessing  an  icon  or  window  at  a  distance  requires  more  cur¬ 
sor  moving,  which  takes  time  and  raises  cursor-tracking  issues  [98]. 
Reaching  distant  information  becomes  even  harder  with  hetereoge- 
neous  monitor  configrations  (e.g.  a  SmartBoard  combined  with  a 
regular  LCD  panel  plus  a  PDA). 

2.  Tracking  the  cursor.  With  increased  physical  screen  size,  users 
employ  higher  mouse  acceleration  to  traverse  large  displays  [8].  The 
faster  the  mouse  cursor  moves,  however,  the  more  difficulty  users 
have  keeping  track  of  it.  In  addition,  during  meetings  or  presenta¬ 
tions  with  large-format  display  facilities,  a  speaker  is  likely  to  point 
a  cursor  at  a  target  to  direct  audience’s  attention.  Locating  a  sta¬ 
tionary  cursor,  however,  becomes  increasingly  problematic  on  large 
displays  [59]. 


3.  Crossing  bezels.  Using  multiple  monitors  is  still  a  popular 
configuration  to  gain  extra  working  space.  Multi-monitors  display 
bezels  are  beneficial  in  allowing  users  to  organize  multiple  tasks  onto 
different  monitors  [98].  Problems  occur,  however,  when  users  cross 
bezels.  A  windows  or  an  image  may  be  sufficiently  large  to  occupy 
several  monitors,  creating  visual  discontinuity  at  bezels.  When  a 
cursor  traverses  across  a  bezel,  there  is  normally  a  discrepancy  be¬ 
tween  its  actual  traveling  course  and  what  users  may  expect,  since 
there  is  no  virtual  space  underneath  the  bezels. 

4.  Managing  space  and  layout.  Interaction  with  large  high- 
resolution  displays  imposes  many  space  and  layout  management  is¬ 
sues,  especially  when  windowing  systems  are  used.  On  desktop  dis¬ 
plays,  various  window  or  task  management  systems  exist,  such  as 
Apple  Expose  [3].  Effectively  handling  space  and  layout  on  large- 
format  displays,  however,  is  by  no  means  trivial. 

5.  Transitioning  between  interactions.  Based  on  tasks  a  user  is 
likely  to  perform,  interaction  with  large  displays  can  be  categorized 
into  two  broad  paradigms.  For  tasks  involving  dealing  with  detailed 
information,  working  up  close  to  a  large  display  is  reasonable.  There 
are  other  tasks,  however,  that  are  best  performed  from  a  distance, 
such  as  sorting  photos  and  pages  or  presenting  large  drawings  to 
a  group  [126].  Also,  large  displays  often  feature  touch  screen  ca¬ 
pacity.  Consequently,  techniques  allowing  a  smooth  transition  from 
up-close  interaction  to  interaction  at  a  distance  and  vice  versa  are 
needed. 

6.2  Extending  Existing  GUIs  and  2D  Metaphors 

Traditional  desktop  metaphors  such  as  Windows,  Icons,  Menus,  and 
Pointing  (WIMP)  do  not  always  scale  well  to  large  format  displays. 
Even  for  large  displays  that  are  still  situated  on  physical  desktops, 
the  amount  of  screen  real  estate  can  make  pointing  to  or  even  find¬ 
ing  windows,  icons,  and  menus  difficult.  Baudisch  et  al.  [8]  found 
that  a  high  density  of  cursor  positions  seen  on  the  screen  could  as¬ 
sist  users  in  visualizing  the  mouse  path,  which  resulted  in  a  small 
improvement  of  performance  in  moving  to  distant  targets  and  higher 
user  satisfaction.  Robertson  et  al.  [98]  propose  a  set  of  extensions 
to  the  traditional  mouse  actions  and  window  capabilities  that  aim  to 
enable  the  user  to  find,  move,  and  manage  desktop  objects  more  eas¬ 
ily.  They  also  address  issues  such  as  the  confusion  caused  by  the 
bezel  between  physical  displays,  which  limits  usability  for  systems 
that  simply  place  multiple  displays  in  proximity. 

A  number  of  techniques  allow  users  to  share  portions  of  win¬ 
dows  [11,  98],  which  can  be  quite  useful  for  moving  tools  near  areas 
in  large-format  displays  where  they  can  be  helpful,  or  for  briefly 
viewing  windows  at  remote  locations  of  the  desktop  without  having 
to  move  to  a  new  location  and  lose  context.  Another  set  of  tech¬ 
niques  helps  draw  the  user’s  attention  to  specific  areas  of  the  display 
by  changing  the  relative  illumination  of  items  of  interest  [9].  In  a 
search- and-identify  task,  Spotlight  [59]  was  found  to  significantly 
increase  the  user’s  speed  in  identifying  the  desired  information.  The 
application  may  also  give  users  the  ability  to  manually  move  win¬ 
dows  large  distances  across  the  display  [39]. 

Drag-and-drop  is  a  popular  technique  for  passing  information  be¬ 
tween  applications  (including  the  operating  system  and  an  applica¬ 
tion)  or  collecting  items  in  desktop  interfaces.  Since  this  technique 
requires  the  pointing  action  in  traditional  WIMP  interfaces,  it  suffers 
on  large-format  displays.  Several  attempts  have  been  made  to  allevi¬ 
ate  the  difficulties,  notably  moving  potential  targets  near  the  selected 
object  [7,  21].  Such  techniques  improve  user  reach  and  time  over 
drag-and-drop  and  have  competitive  or  lower  error  and  drop  rates. 
It  is  interesting  to  note  that  users  sometimes  performed  faster  with 
traditional  drag-and-drop  over  short  distances,  reflecting  the  famil¬ 
iarity  with  that  technique  and  the  time  for  the  user  to  orient  to  the 
assembled  candidate  targets. 


A  technique  called  Vacuum  [10]  pulls  objects  towards  the  user’s 
invocation  point  so  that  they  may  enter  the  user’s  physical  reach. 
Upon  release,  the  desktop  returns  to  its  previous  (pre-vacuumed) 
state.  This  technique  increased  speed  in  tasks  that  consisted  of  se¬ 
lecting  multiple  targets,  but  not  for  the  task  of  selecting  a  single 
target,  with  an  increase  in  error  rate  of  under  three  times,  mostly  due 
to  missing  small  representations  of  objects  that  should  have  been 
selected. 

Another  novel  GUI  widget  and  interaction  technique,  known  as  a 
“Frisbee”  [58],  addresses  the  problem  of  arm’s  length  direct  manipu¬ 
lation  of  inaccessible  regions  of  large-format  displays  in  an  applica¬ 
tion  independent  way.  A  frisbee  consists  of  a  local  “telescope”  and 
a  remote  “target”,  acting  as  a  portal  to  another  part  of  a  display.  In¬ 
teractions  performed  within  the  telescope  are  applied  on  the  remote 
data,  which  are  surrounded  by  the  target  and  drawn  in  the  telescope. 
This  technique  satisfies  major  design  principles  such  as  minimizing 
physical  travel  and  visual  disruption,  maintaining  visual  consistency, 
and  supporting  concurrent  multi-user  interactions,  and  proves  to  be 
advantageous  over  conventional  “click- walk-click”  interaction  style 
for  wall- sized  displays. 

Direct  drawing  on  2D  displays  is  not  yet  a  standard  interface  tool, 
but  it  can  be  a  powerful  metaphor  for  a  wall-sized  display.  The  Inter¬ 
active  Mural  used  a  pen-based  drawing  tool  [44]  with  which  the  user 
could  draw,  write  text,  or  manipulate  parameters  such  as  the  scale 
and  position  of  the  display  space. 

6.3  3D  User  Interfaces  for  Large  Displays 

General  VR  tracking  systems,  whether  based  on  magnetic,  mechan¬ 
ical,  optical,  ultrasonic,  or  other  technologies,  have  often  been  used 
as  3D  input  devices  for  large  display  spaces.  Through  their  ability 
to  enable  pointing  via  ray-casting  and  similar  techniques,  they  can 
be  used  to  select  objects  at  arbitrary  distances  from  the  user  in  a  3D 
space.  With  their  six  DOF  input,  they  can  be  mapped  to  naviga¬ 
tion  operations  in  3D;  many  such  interfaces  have  been  inspired  by 
helicopter  controls. 

The  Interaction  Table  [45]  provides  six  DOF  input  with  a  series  of 
devices  that  together  can  provide  an  interface  similar  to  a  3D  wheel 
for  navigation  or  manipulation.  This  can  provide  a  natural  interface 
and  has  the  benefit  of  providing  passive  feedback  to  the  user. 

Grossman  et  al.  [42]  developed  a  3D  modeling  interface  for  large- 
scale  displays.  The  interface  integrates  a  variety  of  methods  that 
work  well  for  large-scale  interaction.  Some  of  these  include  2D  con¬ 
struction  planes  spatially  organized  in  a  3D  volume,  tape  drawing  for 
curve  and  line  creation,  and  continuous  two-handed  interaction. 

The  Vision  Wand  [16]  represents  a  compromise  between  the  dif¬ 
ficult  task  of  tracking  hands  and  the  ease  of  users  forming  gestures. 
Using  a  wand  with  colored  endpoints  creates  a  device  that  is  easy  to 
track,  but  requires  little  hardware  in  the  user’s  hands.  The  user  can 
then  gesture  with  such  methods  as  tapping,  tilting,  rotating,  or  mov¬ 
ing  the  wand  to  control  parameters.  Gestures  can  control  standard 
user  interface  items  such  as  menus  and  widgets  to  enable  the  user  to 
manipulate  more  parameters. 

LaViola  et  al.  [65]  developed  a  set  of  novel  input  devices  for 
CAVE-based  virtual  environments.  They  employ  hand  gestures, 
such  as  pointing  with  a  tracked,  finger- worn  sleeve  or  foot  gestures, 
such  a  tapping  toes  or  heels  on  a  map  with  a  foot- worn  slipper  for 
navigation.  They  enable  object  selection  with  flexing  and  pinching 
while  wearing  a  glove  with  buttons  added  to  engage  actions.  Similar 
user  actions  can  be  measured  by  computer  vision  systems  [52,  60], 
but  the  robustness  of  such  systems  still  presents  difficulties  such  as 
incorrectly  recognized  gestures. 

Malik  et  al.  [72]  use  multi-hand  gestures  to  enable  the  user  to 
control  an  object  and  the  workspace  simultaneously,  thus  allowing 
the  user  to  bridge  the  distance  between  objects,  or  to  offer  the  user  a 
wider  range  of  gestures  by  allowing  the  two  hands  to  work  together. 


They  activate  widgets  under  the  hands  to  similarly  add  to  the  power 
of  gesture-based  interactions. 

Vogel  et  al.  [126]  use  hand  gestures  to  indicate  typical  user  in¬ 
terface  actions  such  as  point-and-click  when  working  at  a  distance 
from  the  display  surface.  In  order  to  achieve  a  stable  pointing  opera¬ 
tion,  they  filter  the  detected  finger  position.  They  found  that  closing 
the  thumb  to  the  index  finger  does  not  appear  to  have  any  speed 
or  accuracy  benefit  over  tapping  in  mid-air,  despite  the  kinestethic 
feedback  offered  by  the  former.  They  found  that  a  ray-based  point¬ 
ing  operation  was  faster  but  inaccurate  compared  to  a  technique  that 
combined  a  clutching  gesture  to  engage  control  and  hand  motion  to 
move.  The  latter  technique  enabled  similar  performance  to  a  hybrid 
approach  that  used  approximate  pointing. 

The  Interactive  Workspaces  Project  [57]  explored  interface  possi¬ 
bilities  for  people  working  together  using  large  displays.  They  inte¬ 
grated  a  variety  of  interaction  devices  and  techniques  including  wire¬ 
less  multimodal  devices.  Other  similar  work  combining  interaction 
devices  with  display  walls  can  be  found  in  [97,  92,  75,  53,  38,  82]. 

6.4  User  Interface  Evaluation 

While  many  believe  that  large-format  displays  automatically  pro¬ 
vide  benefits,  the  evidence  is  not  quite  so  clear.  Tyndiuk  et  al.  [120] 
found  that  navigation  to  objects  that  are  already  in  view  does  not 
benefit  from  a  large  display  over  a  standard  desktop  display,  whereas 
navigation  to  unseen  objects  does  (presumably  due  to  benefit  in  the 
search  portion  of  the  navigation).  They  also  found  that  performance 
on  a  manipulation  task  improved  with  the  large  display  over  the  stan¬ 
dard  desktop.  They  further  found  a  correlation  between  visual  at¬ 
tention  and  performance  with  large  displays  compared  to  small  dis¬ 
plays. 

7  Top  Ten  Research  Challenges 

After  consideration  of  the  issues  associated  with  large  high- 
resolution  displays,  we  have  come  up  with  a  list  of  what  we  believe 
are  the  top  ten  research  challenges  faced  by  this  community.  We 
hope  the  challenges  inspire  future  research  projects  involving  large 
high-resolution  displays. 

1.  Truly  seamless  tiled  displays.  Tiling  projected  images  to  form 
a  ’’seamless”  large  high-resolution  display  has  been  a  popular  ap¬ 
proach,  and  we  have  witnessed  a  lot  of  work  done  on  image  blend¬ 
ing  and  geometric  registration.  However,  unresolved  technological 
problems  exist,  such  as  variations  of  color  and  luminosity,  which 
may  easily  break  the  illusion  of  a  single  seamless  display.  Calibra¬ 
tion  is  also  reported  to  be  a  headache  in  practice. 

2.  Stereoscopic  large  high-resolution  displays.  Building  large- 
scale,  high-resolution  headtracked  stereoscopic  displays  is  a  key 
challenge  to  producing  a  high-resolution  immersive  virtual  environ¬ 
ment  experience.  The  Varrier  autostereoscopic  display  developed  at 
EVL-UIC  [102]  and  a  few  projector-based  high-resolution  solutions 
(e.g.,  Cyviz’s  Vizwall  [24])  are  achievements  heading  in  the  right 
direction  toward  solving  this  challenge. 

3.  Easily  reconfigurable  large  high-resolution  displays.  Creating 
displays  that  can  easily  be  reconfigured  and  support  several  form 
factors  (e.g.,  flat,  curved,  and  other  representations)  is  desirable.  To¬ 
day’s  reconfigurable  displays  often  require  tedious  hours  of  realign¬ 
ment  after  any  shape  reconfiguration.  Future  reconfigurable  displays 
should  be  easy  to  pack,  move,  align,  and  color  calibrate. 

4.  High-performance  cluster  rendering.  There  is  a  growing  num¬ 
ber  of  software  APIs  and  toolkits  for  cluster  rendering  with  sup¬ 
port  for  high-resolution  displays.  A  number  of  toolkits  support  only 
master-slave  data  distribution.  These  schemes  are  easy  to  imple¬ 
ment  and  allow  legacy  applications  that  can’t  be  parallelized  to  run 


in  a  cluster  environment.  However,  they  do  not  fully  utilize  cluster 
resources  since  every  node  has  to  run  a  copy  of  the  application  and 
there  is  no  performance  increase  compared  with  running  the  applica¬ 
tion  on  a  single  machine.  Some  toolkits  intercept  OpenGL  function 
calls  and  distribute  rendering  data  to  cluster  nodes.  While  this  is 
transparent  to  the  application,  these  toolkits  require  high-bandwidth 
and  low-latency  connections  between  cluster  nodes.  There  is  no  ul¬ 
timate  solution  for  cluster  rendering  today,  and  the  choice  for  a  par¬ 
ticular  toolkit  largely  depends  on  the  application. 

5.  Scalability.  The  majority  of  tiled-display  installations  today  is 
limited  to  fewer  than  twenty  tiles.  Existing  cluster  rendering  and 
display  data  streaming  software  provides  adequate  support  for  these 
types  of  systems.  However,  we  expect  new  challenges  with  the  ad¬ 
vance  of  massively- tiled  displays  such  as  EVL-UIC’s  55-tile  Lam- 
daVision  system. 

6.  Design  and  evaluate  large  high-resolution  display  groupware. 

Although  large-format  displays  are  appealing  technologies  to  sup¬ 
port  collaborative  interaction,  it  remains  challenging  to  design  and 
evaluate  groupware  applications  that  fully  exploit  their  capacity  and 
potential  [49].  Grudin  outlined  research  challenges  for  creating 
desktop  groupware  applications  in  [43],  most  of  which  still  hold  up 
in  designing  groupware  applications  for  large  high-resolution  dis¬ 
plays.  However,  unique  characteristics  and  requirements  of  large- 
scale  display  groupware  systems  present  new  challenges.  For  exam¬ 
ple,  users  perceive  large  high-resolution  displays  in  radically  differ¬ 
ent  ways  due  to  their  form  factors,  which  may  affect  the  groupware 
interface  design.  Also,  large  display  groupware  applications  have 
to  accomodate  semi-public  or  public  contexts,  which  may  affect  the 
visibility  and  privacy  of  interactions  [49]. 

7.  Effective  interaction  techniques.  Traditional  mouse  and  key¬ 
board  and  associated  standard  desktop  interaction  techniques  are  not 
sufficient  for  interaction  with  large-format  displays.  A  number  of 
interaction  techniques  have  been  investigated  for  large  displays  in¬ 
cluding  natural  gestures,  voice  recognition,  multi-handed  interaction 
techniques,  and  methods  to  improve  the  reach  of  the  user.  Also,  sev¬ 
eral  user  interface  metaphors  for  facilitating  specific  tasks  such  as 
windows  management  and  distal  target  access  have  been  explored. 
These  techniques  show  promise,  but  need  to  be  evaluated  for  spe¬ 
cific  tasks  in  order  to  gain  a  better  understanding  of  how  effective 
they  are  for  interacting  with  large  displays. 

8.  Perceptually  valid  ways  of  presenting  information  on  the  large 
displays.  The  field  of  view  has  a  demonstrable  effect  on  the  percep¬ 
tion  of  the  user;  this  is  the  most  obvious  but  not  the  only  feature  of 
perceptual  significance  for  large  displays.  Factors  such  as  apparent 
brightness,  contrast,  and  resolution  heavily  affect  the  user’s  under¬ 
standing  of  information  on  a  display.  Non-visual  effects,  such  as 
ergonomic  comfort,  may  also  play  a  role. 

9.  Empirical  evidence  for  the  benefits  of  large  high-resolution 
displays.  A  taxonomy  that  matches  low-level  tasks,  perceptual  fac¬ 
tors,  and  high-level  applications  with  different  form-factor  display 
types  (e.g.,  resolution,  physical  size,  configuration)  would  enable 
laboratories  to  gain  the  most  benefit  for  their  applications  without 
having  to  risk  investing  significant  resources  on  less  appropriate  sys¬ 
tems.  We  should  analyze  the  benefits  and  limitations  of  large  high 
resolution  for  a  range  of  tasks.  We  have  presented  some  work  in  this 
area,  but  much  more  is  needed. 

10.  Integrating  large  high-resolution  displays  into  a  seamless 
computing  environment.  It  is  an  increasing  trend  to  bring  portable 
computing  devices  into  workspaces.  A  seamless  computing  environ¬ 
ment  is  an  infrastructure  supporting  a  smooth  integration  of  portable 
devices  and  pre-installed  computing  facilities,  making  it  possible  to 
fluidly  exchange,  share,  and  store  information  within  a  collaborative 


group.  While  large  high-resolution  displays  are  becoming  preva¬ 
lent  to  support  collaborative  tasks,  a  challenging  question  arises:  is 
there  an  effective  way  to  seamlessly  integrate  emerging  display  tech¬ 
nologies  with  existing  heterogeneous  devices?  Associated  are  many 
fundamental  research  issues  to  address  including  low  level  network¬ 
ing  and  hardware  interfaces,  supporting  software  toolkits,  user  inter¬ 
faces,  and  social  interaction. 

8  Conclusions 

We  have  presented  a  comprehensive  survey  of  prior  research  on  large 
high-resolution  displays,  identified  major  unresolved  problems,  and 
from  these  made  a  list  of  what  we  feel  are  the  top  ten  challenges  in 
the  field.  Our  expectation  is  that  the  survey  and  top  ten  challenges 
will  bring  attention  to  what  has  been  done  and  what  needs  to  be  done 
for  research  involving  large  high-resolution  displays.  We  hope  to 
inspire  the  display  community  into  having  more  discussion  and  de¬ 
bates  about  research  in  this  field,  and  more  importantly,  spark  efforts 
to  maintain  an  updated  list  of  achievements  and  current  research 
challenges  in  the  field.  Scientific  research  for  large  high-resolution 
displays  has  been  advancing  tremendously  in  the  past  decade,  and 
we  believe  it  is  far  from  running  out  of  steam.  We  feel  it  also  will 
evolve  into  an  interdisciplinary  research  area  with  a  stimulating  and 
challenging  agenda  in  the  future. 
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Figure  Cl:  Los  Alamos  National  Laboratory’s  ultra  high-resolution  immersive  room — La  Cueva  Grande  [34] — developed  by  Fakespace  Systems  [33]  (Image 
courtesy  of  Los  Alamos  National  Laboratory,  reference  LA-UR-05-7498.  Photographed  by  Presley  Salaz,  LANL). 

Figure  C2:  Tiled-LCD  multi-monitor  desktop  display  (Image  courtesy  of  Ball  [4]). 

Figure  C3:  LambdaVision,  a  100-MPixel  wall  (Image  provided  by  Luc  Renambot,  Electronic  Visualization  Laboratory,  University  of  Illinois  at  Chicago  [94,  93]). 


Figure  C4:  LambdaTable,  a  horizontal  tiled  display  and  interface  (Image  provided  by  Luc  Renambot,  EVL-UIC  [94,  93]). 

Figure  C5:  Stackable,  reconfigurable  projector  array,  VisBlock™from  VisBox  Innovative  Display  and  Interaction  Technologies  (Image  courtesy  VisBox.com  [124]). 
Figure  C6:  Cyviz’s  stereoscopic  projector  array  called  Cyviz  Vizwall™(lmage  courtesy  Cyviz.com  [24]). 


Figure  C7:  EVL-UIC’s  autostereoscopic  display  called  Varrier™ (Image  courtesy  EVL-UIC  [102]). 

Figure  C8:  Cutaway  mockup  of  DepthCube™display  from  LightSpace  Technologies  (Image  courtesy  LightSpace  Technologies  [112]). 
Figure  C9:  NRL’s  immersive  room  demonstrating  a  submarine  command  &  control  application. 


Figure  CIO:  Deere  &  Company’s  VR  testing  facility  for  virtual  prototyping  vehicle  designs  and  testing  their  effectiveness  (Photo  courtesy  Deere  &  Company, 
Moline,  Illinois  [27]). 

Figure  Cll:  State-of-the-art  meeting  facility  at  Alias  Visualization  Studio  (Image  courtesy  of  Alias  [2]). 


